Skip to content

HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks #5002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ZanderXu merged 1 commit into from
Oct 25, 2022
Merged

HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks #5002

ZanderXu merged 1 commit into from
Oct 25, 2022

Conversation

@hotcodemacha
Copy link
Contributor

@hotcodemacha hotcodemacha commented Oct 11, 2022

Description of PR

NumAllSinks metrics shows lower value than NumActiveSinks

JIRA - HADOOP-9946

How was this patch tested?

UT

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

ZanderXu
ZanderXu approved these changes Oct 11, 2022
View reviewed changes
Copy link
Contributor

@ZanderXu ZanderXu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ashutoshcipher for your report.

LGTM.

But I have a little different through. Maybe the Map<String, MetricsSink> allSinks is only used to store the registered MetricsSink and sinks is used to store all the MetricsSinkAdapter.

Do this modification or not looks good to me. Let's wait any good suggestions.

@hotcodemacha
Copy link
Contributor Author

hotcodemacha commented Oct 11, 2022

Thanks @ZanderXu for review and approval. @aajisaka is the original reporter for this issue, may be he can help with additional review/suggestions.

@hadoop-yetus
Copy link

hadoop-yetus commented Oct 11, 2022

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 42m 41s trunk passed
+1 💚 compile 26m 0s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 27m 36s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 27s trunk passed
+1 💚 mvnsite 1m 55s trunk passed
+1 💚 javadoc 1m 27s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 58s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 3m 0s trunk passed
+1 💚 shadedclient 26m 21s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 6s the patch passed
+1 💚 compile 24m 54s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 24m 54s the patch passed
+1 💚 compile 22m 26s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 22m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 19s the patch passed
+1 💚 mvnsite 1m 53s the patch passed
+1 💚 javadoc 1m 18s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 58s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 2m 59s the patch passed
+1 💚 shadedclient 26m 1s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 18m 39s hadoop-common in the patch passed.
+1 💚 asflicense 1m 10s The patch does not generate ASF License warnings.
235m 53s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/artifact/out/Dockerfile
GITHUB PR #5002
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a7442b5eaeae 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 37a3024
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/testReport/
Max. process+thread count 3144 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hotcodemacha
Copy link
Contributor Author

hotcodemacha commented Oct 20, 2022 •
edited
Loading

@ZanderXu - Can we commit/merge this? (We dont have any other comments/suggestions)

Thanks

aajisaka
aajisaka approved these changes Oct 25, 2022
View reviewed changes
Copy link
Member

@aajisaka aajisaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thank you @ashutoshcipher for bringing this up, and thank you @ZanderXu for your review.

@ZanderXu ZanderXu merged commit 0a26d84 into apache:trunk Oct 25, 2022
@ZanderXu
Copy link
Contributor

ZanderXu commented Oct 25, 2022

Merged. Thanks @ashutoshcipher for your contribution and thanks @aajisaka for your review.

@aajisaka
Copy link
Member

aajisaka commented Oct 25, 2022

But I have a little different through. Maybe the Map<String, MetricsSink> allSinks is only used to store the registered MetricsSink and sinks is used to store all the MetricsSinkAdapter.

Good point. I suppose we can simplify allSinks because it's only used to avoid duplicate sink names. Set<String> maybe sufficient instead of Map<String, MetricsSink>.

    if (allSinks.containsKey(name)) {
      LOG.warn("Sink "+ name +" already exists!");
      return sink;
    }

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@hotcodemacha
HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks (
bd8ce1d 
apache#5002)

Reviewed-by: Akira Ajisaka <[email protected]>
@slfan1989 slfan1989 mentioned this pull request Jan 15, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aajisaka aajisaka aajisaka approved these changes

@ZanderXu ZanderXu ZanderXu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hotcodemacha @hadoop-yetus @ZanderXu @aajisaka